Cone-culled soft shadows

ABSTRACT

Soft shadows in computer graphics images are created by rendering the scene from the camera viewpoint and at least one light viewpoint. The positions of scene fragments and light fragments in the scene are stored. For each scene fragment, a frustum is defined between the position of the scene fragment and the light source. Light fragments are evaluated with respect to the frustum to select light fragments blocking light between the light source and the scene fragment. A color or monochromatic shading value is determined for each scene fragment that indicates the amount of light blocked or transmitted by the light fragments. The shading values are then used to alter scene fragments accordingly. Computer graphics images with soft shadows can be created entirely by a graphics processing subsystem or by a graphics processing subsystem in conjunction with a central processing unit using a pipelined, deferred shading approach.

BACKGROUND OF THE INVENTION

The present invention relates to the field of computer graphics. Manycomputer graphic images are created by mathematically modeling theinteraction of light with a three dimensional scene from a givenviewpoint. This process, called rendering, generates a two-dimensionalimage of the scene from the given viewpoint, and is analogous to takinga photograph of a real-world scene. A great deal of effort has beendevoted to generating realistic and artistically compelling computergraphics images and animation.

Shadows are among the most important psychophysical clues for judgingthe relative size, positions, and shapes of objects. Shadows alsoenhance the mood and visual impact of scenes.

There are numerous algorithms and rendering techniques for generatingshadows in computer graphics images. Ray tracing and global illuminationrendering algorithms can create images with realistic looking shadows.However, these two rendering algorithms are computationally expensiveand typically cannot be used for real-time rendering applications.

Real-time rendering algorithms, such as rasterization algorithms, arecommonly used to create computer graphics images at interactive framerates. Real-time rendering algorithms can be implemented in softwareexecuted by a general purpose central processing unit (CPU) andoptionally a graphics processing subsystem (GPU), which includeshardware specifically adapted to create computer graphics images byperforming operations such as geometry tessellation; transformation;clipping; rasterization; lighting, shading, and texture mappingoperations; occlusion and z-buffer evaluation; and/or blending andcompositing operations.

Real-time rendering algorithms can apply shadow volume and/or shadowmapping techniques to create shadows. Typical shadow volume and shadowmapping techniques tend to produce shadows with abrupt, sharply definedboundaries between the lit and shadowed areas. However, real-world lightsources typically produce shadows with soft or partially shadowedboundaries between the lit and shadowed areas, referred to soft shadows.As a result, the shadows generated by typical real-time renderingalgorithms look harsh and unrealistic.

Some improved real-time shadow algorithms use image filters, multiplesamples of rendering data, and/or multiple rendering passes to generatesoft shadows. However, these techniques are time-consuming, low quality,and/or require extensive pre-processing, making them unacceptable forreal-time rendering of dynamic scenes. Moreover, these algorithms cannotcorrectly render shadows cast by translucent objects or colored shadows,which can be introduced by colored light sources.

It is therefore desirable for a system and method to efficiently createrealistic looking soft shadows in computer graphics images that aresuitable for use with real-time rendering algorithms. It is furtherdesirable for the system and method to generate soft shadows without theneed for pre-processing, making it suitable for use with dynamic scenes.It is also desirable for the system and method to be capable ofgenerating soft shadows for translucent objects and colored shadows. Itis still further desirable for the system and method to be readilyadaptable for efficient implementation in conventional computers systemsand/or graphics processing subsystems, as well as multithreaded,multiprocessor, and/or multiple core parallel processing systems.

BRIEF SUMMARY OF THE INVENTION

An embodiment of the invention can create realistic-looking and dynamicsoft shadows in computer graphics images. In an embodiment, the scene isrendered from the camera viewpoint and the positions of scene fragmentsin the scene are stored in a first buffer. The scene is also renderedfrom the viewpoint of at least one light, and the positions of lightfragments in the scene are stored in a second buffer. For each scenefragment, a frustum is defined between the position of the scenefragment and the light source. Light fragments in the second buffer areevaluated with respect to the frustum to select light fragments blockinglight between the light source and the scene fragment. A shading valueis determined for each scene fragment based on its respective selectedlight fragments that indicates the amount of light blocked by the lightfragments. The shading values are then used to lighten or darken scenefragments accordingly. Embodiments of the invention can be executedentirely by a graphics processing system or a graphics processing systemin conjunction with a central processing unit using a pipelined,deferred shading approach.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the drawings, inwhich:

FIG. 1 illustrates a method of generating soft shadows according to anembodiment of the invention;

FIGS. 2A-2E illustrate example applications of the steps of a method ofgenerating soft-shadows according to an embodiment of the invention;

FIG. 3 illustrates an example application of depth peeling suitable foruse with an embodiment of the invention;

FIGS. 4A-4B illustrate implementations of a method of generating softshadows according to embodiments of the invention;

FIG. 5 illustrates details of an implementation of a method ofgenerating soft shadows according to an embodiment of the invention;

FIG. 6 illustrates an example hardware system suitable for implementingan embodiment of the invention; and

FIG. 7 illustrates an example processor suitable for implementing anembodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates a method 100 of generating soft shadows according toan embodiment of the invention. Step 105 renders a scene from theposition of a virtual “camera” or viewer. Step 105 can employ anyrendering technique known in the art, such as ray tracing,rasterization, scanline rendering, or deferred rendering or shadingtechniques. Moreover, the scene can be defined using any technique knownin the art for defining three-dimensional geometry, including points,lines, polygons, quadratic or higher order surfaces, and subdivisionsurfaces.

Step 105 generates a set of fragments representing the portion of thescene visible from the point of view of the camera. Each fragmentrepresents a sample of the scene. This set of fragments is stored in adata structure referred to as the eye buffer. Each fragment in the eyebuffer is associated with an image space position representing thelocation of the fragment projected onto an image plane specified by thecamera. In embodiments of method 100, background portions of the scenesare either not included in the eye buffer or marked as backgroundfragments. Background portions of the scene can include fragmentsassociated with the empty space or sky, distant objects that do notaffect the shading of objects near the camera, or any other objects orportions of the scene that are to be excluded from the application ofthese shadow generation techniques, such as portions to be shaded usingother shading techniques. In alternate embodiments, fragments to beshadowed can be created and identified in other techniques independentof the viewpoint.

In an embodiment, step 105 determines attributes for each fragment inthe eye buffer including the position of each fragment in the worldspace coordinate system (or any other convenient scene-relatedcoordinate system) and an opacity value of the fragment. In embodiments,the opacity value can be a scalar value representing the fragment'sopacity to white light or a vector value representing the fragment'sopacity to different colors of light. In some implementations, step 105sets the opacity value for empty eye buffer elements to zero.

As discussed in detail below, step 105 can also determine any otherfragment attributes necessary to determine the visual attributes of thefinal rendered image, such as color and texture mapping values. Theseadditional attributes can be generated using any rendering techniqueknown in the art and are unimportant for creating soft shadows inaccordance with embodiments of the invention. Although not used forgenerating soft shadows, these additional attributes can be stored inthe eye buffer for convenience or alternatively in a separate datastructure.

In an embodiment, the image space position of each fragment may beimplicit in the data structure format of the eye buffer. For example, inan M by N size eye buffer, the first element of the eye buffer storesthe fragment projected onto the image space position of (0,0), the Mthelement of the eye buffer stores the fragment projected onto the imagespace position of (M−1,0), and the M*Nth element of the eye bufferstores the fragment projected onto the image space position of(M−1,N−1).

In another embodiment, the eye buffer may arrange fragment attributes ina packed data format suitable for processing by single instruction,multiple data (SIMD) operations. For example, if each fragment includesX, Y, and Z position attributes, the attributes of four fragments (F1,F2, F3, and F4) may be stored in eye buffer elements in the followingorder: X_(F1), X_(F2), X_(F3), X_(F4), Y_(F1), Y_(F2), Y_(F3), Y_(F4),Z_(F1), Z_(F2), Z_(F3), and Z_(F4). This example arrangement facilitatesthe processing of four fragments in parallel using SIMD operations.

Step 110 generates a set of light fragments representing the portion ofthe scene visible from the point of view of a light source that isdesignated to cast soft shadows. Each light fragment represents a sampleof the scene from the point of view of the light source. This set oflight fragments is stored in a data structure referred to as the lightbuffer. Each light fragment in the light buffer is associated with animage space position representing the location of the light fragmentprojected onto an image plane specified by the point of view of thelight source. Similar to step 105, step 110 may exclude or markbackground fragments in the light buffer to prevent these backgroundportions of the scene from casting shadows.

In an embodiment, step 110 determines attributes for each light fragmentin the light buffer including the position of each light fragment in theworld space coordinate system (or any other convenient scene-relatedcoordinate system). In a further embodiment, step 110 may also determinea light intensity value for each light fragment, representing the lightintensity and optionally the color of light at the position of the lightfragment. In another embodiment, step 110 instead determines a shadowintensity for each light fragment, which is the attenuation of lightfrom a maximum value, for example based on the distance of the lightfragment to the light source. In a further embodiment, the lightintensity (or shadow intensity) can be modified by a light fragment'stransparency value, which arises when the light fragment is part of atransparent or translucent object casting a shadow. The light intensityvalue can be a scalar to provide monochromatic shadows or a vector(including for example separate red, green, and blue intensity values)to provide color shadows. The data structure of the light buffer can besimilar to that discussed above with respect to the eye buffer.

A light source can be directional light, such as a spotlight, or anomni-directional light. Light sources have a size and/or shape, whichdetermines the size of the penumbra or partially shadowed region cast bythe light source. (Infinitesimally small or point light sources castsharp shadows). For directional lights, an embodiment of step 110defines a view frustum and light buffer image plane based on thedirection, size, and area of effect of the directional light source. Foromni-directional light sources, an embodiment of step 110 defines a viewfrustum and light buffer image plane based on the size of the lightsource and the position of the camera or viewer, such that the lightsource's view frustum covers the camera or viewer's field of view and/orany objects potentially casting shadows within the camera or viewer'sfield of view. An alternate embodiment of step 110 can use a fish-eye orother omni-directional projection to generate a light buffer withfragments in every direction surrounding the light source.

Following the generation of fragments in the eye buffer and lightbuffer, step 115 selects a fragment in the eye buffer for processing. Inan embodiment, only non-background fragments in the eye buffer areselected. Step 120 defines a frustum for the selected fragment. In anembodiment, step 120 defines the frustum between the selected fragment'sposition in world space, as stored in the eye buffer, and the boundariesof the light source, as determined by the size and/or shape of the lightsource. In an embodiment, the frustum can be defined in part by theexact boundaries and shape of the light source. In another embodiment,the light source's boundaries can be approximated by a bounding sphere.In this embodiment, the frustum for the selected fragment can be definedas a cone with a tip or apex located at the selected fragment's positionin world space and a base corresponding with a cross-section of thelight source's bounding sphere.

Step 125 selects light fragments inside the frustum of the selectedfragment. In an embodiment, step 125 projects the selected fragment'sfrustum into the light source's image plane. Step 125 selects all or aportion of the light fragments in the light buffer that, as projected inthe light buffer, reside inside the two-dimensional region defined byprojecting the frustum in the plane of the light buffer. However, evenif a light fragment projected into the light buffer is inside thetwo-dimensional region of the projected frustum, the light fragment'sposition in world space may be outside the frustum itself. Thus, step125 further evaluates each selected light fragment's world spaceposition against the boundaries of the frustum to determine if the lightfragment is inside the frustum.

In an embodiment, step 125 determines the projection of the frustum andselects all of the light fragments residing inside the region of theprojection for further evaluation with the frustum itself. In anembodiment, light fragments outside the projected frustum include emptyentries or background fragments of the light buffer, which correspondwith portions of the scene unaffected by the light source.

In another embodiment, step 125 determines a bounding box or region forthe projected frustum and selects all or a portion of the lightfragments residing inside this bounding box region for furtherevaluation with the frustum itself. In still another embodiment, step125 defines a major axis of the frustum as the line between the selectedfragment of the eye buffer in world space and the center of the lightsource in world space and projects the major axis onto the light imageplane. Step 125 then selects all or a portion of the light fragmentsalong or near the major axis for further evaluation with the frustumitself.

Following step 125, a set of all or a representative portion of thelight fragments having world space positions within the frustum of theselected fragment will be used to determine the illumination value ofthe selected fragment.

Step 130 determines an illumination value for the selected fragment ofthe eye buffer by evaluating the set of light fragments selected in step125. The intensity values of the selected light fragments represent (orapproximate) the amount of light from the light source that is blockedfrom the selected fragment of the eye buffer.

In an embodiment, step 130 sums the intensity values of the selectedlight fragments to determine an illumination value for the selectedfragment of the eye buffer. In another embodiment, it is assumed thateach light fragment's intensity value represents the amount of lightattenuated over a volume of space between the light source and the lightfragment, and thus step 130 multiplies the selected light fragments'intensities to determine an illumination value for the selected fragmentof the eye buffer. In still a further embodiment, step 130 may normalizeor scale the illumination value of the selected fragment of the eyebuffer based on the total number of selected light fragments or based onthe ratio of selected light fragments to the total number of lightfragments in the projected frustum or any approximation thereof, such asa bounding box. In this case of colored lights and shadows, illuminationvalues can be determined by processing light fragment intensity valueson a per color or component basis.

Step 135 determines a shaded value of the selected fragment in the eyebuffer. Typically, the shaded value of the selected fragment includescolor information (e.g. red, green, and blue values). In an embodiment,a renderer determines the shaded value of the selected fragment of theeye buffer from the fragment's illumination value and other fragmentattributes used in rendering, such as color, transparency, reflectivity,surface normal vectors, and texture mapping coordinates, as well as theexecution of one or more shading programs. For example, step 135 canblend, for example by scaling, the unshaded value of the selectedfragment with the fragment's illumination value.

Following step 135, method 100 returns to step 115 to select anotherfragment in the eye buffer for processing. Once all of the fragments inthe eye buffer have been processed, the contents of the eye buffer areready to be displayed or further processed by other renderingoperations. In a further embodiment, method 100 can be repeated foradditional light sources. In another embodiment, each light source isevaluated several times with the depth of near clipping plane of thelight source view frustum changed, in a process referred to as depthpeeling. Depth peeling provides improved shadow quality when renderingobjects with complex light occlusion and scattering properties, such asa model of a tree with thousands of leaves. In further embodiments, theresults of step 105 can be cached and reused for multiple light sourcesand/or depth layers. In additional embodiments, the light bufferscreated by step 110 can be cached for reuse in additional frames if thelights and occluders are static or change minimally.

In another embodiment, all of the other attributes of the selectedfragment of the eye buffer are processed prior to step 135, for examplein step 105 or in parallel with method 100, to determine an unshadedvalue of the selected fragment, which may be stored in the eye buffer ora different buffer, such as an unshaded image buffer. Step 130 may alsocreate a shading buffer storing the illumination values for all of thefragments evaluated from the eye buffer. Step 135 can then blend theshading buffer with the unshaded image buffer on a per-fragment basis todetermine a shadowed and optionally shaded image.

Moreover, as discussed in detail below, steps 115, 120, 125, 130, and135 can be performed for multiple fragments in parallel using, forexample, single instruction single data (SISD) operations or SIMDoperations executed by one or more processors or processor cores.

FIGS. 2A-2E illustrate example applications of the steps of a method ofgenerating soft-shadows according to an embodiment of the invention.FIG. 2A illustrates an example application 200 of step 105 creatingfragments for an eye buffer. Example 200 includes a virtual camera 202representing the point of view of a viewer of a scene. A scene isrendered from the position of the virtual camera 202 to generatefragments in eye buffer 204.

As discussed above, the eye buffer 204 stores attributes of fragmentsincluding the fragments' positions in a world space coordinate system.For example, eye buffer 204 includes entries 206A, 206B, and 206C. Eyebuffer entries 206A, 206B, and 206C correspond with fragments 208A,208B, and 208C, respectively, of an object 207 projected onto the planeof the eye buffer 204. As shown in FIG. 2A, fragments 208A, 208B, and208C have positions in world space.

As discussed in detail below, this example 200 includes at least onelight source 212, which has a size represented by bounding sphere 214,and at least one object, referred to as an occluder 210, casting ashadow from the light source 212.

FIG. 2B illustrates an example application 215 of step 110 creatinglight fragments for a light buffer 216. In this example, the scene isrendered from the position of the light source 212 to generate lightfragments for the light buffer 216. As discussed above, the light buffer216 stores attributes of light fragments, including the light fragments'positions in a world space coordinate system and intensity valuesrepresenting the amounts of light received or blocked by the lightfragments. For example, light buffer 216 includes entries 218A, 218B,and 218C. Light buffer entries 218A, 218B, and 218C correspond withlight fragments 220A, 220B, and 220C, respectively, which are fragmentsof the occluder 210 and the object 207 that receive light from lightsource 212. Although example 215 illustrates a single occluder 210,fragments from any number of occluders can be included in the lightbuffer 216 with no or only a nominal increase in processing time. Asshown in FIG. 2B, light fragments 220A, 220B, and 220C have positions inworld space.

FIG. 2C illustrates an example application 225 of steps 115, 120, and125. In example 225, an entry 226 of the eye buffer 204 is selected andits corresponding fragment 228 in world space is identified. A frustum230 is defined between the location of fragment 228 in world space andthe boundaries 214 of the light source 212. In an embodiment, thisfrustum 230 forms a cone. However, in alternate embodiments, othershapes such as tetrahedrons can be used.

The frustum 230 is projected into the plane of the light buffer 216.This projection defines a region 231 in which all or a portion of thelight fragments in light buffer will be evaluated to determine theillumination value of fragment 228. For example, light fragments 232Aand 232B fall within region 231 and may potentially contribute to theillumination value of the fragment 228. Conversely, light fragments 234Aand 234B fall outside the region 231 and do not affect fragment 228.

FIG. 2D illustrates the projection of the frustum in the light buffer250 in more detail. Light buffer 250 includes a set of light fragments255, such as light fragments 255A, 255B, 255C, 255D, 255E, and 255F. Theprojection of the frustum defines two-dimensional region 231 in thelight buffer 250. FIG. 2D also shows the selected fragment 228 from theeye buffer projected into the light buffer 250.

In an embodiment, all or some of the light fragments within the region231 may be evaluated to determine the illumination value of the selectedfragment. In another embodiment, an approximate region 265, such as abounding box, is used to approximate the region 231 and all or some ofthe light fragments within this approximate region 265 are evaluated todetermine the illumination value of the selected fragment. In anotherembodiment, all or some of the light fragments on or near the projectionof the major axis 260 of the frustum are evaluated to determine theillumination value of the selected fragment.

A discussed above, a light fragment does not affect the shadowing of aselected fragment from the eye buffer unless the light fragment's worldspace position is inside the three-dimension frustum defined between theselected fragment from the eye buffer and the light source. In anembodiment, the frustum of a fragment is projected into the light bufferto define a two dimensional region, such as projected frustum region 231or approximate projected frustum region 265 in FIG. 2D. All or a portionof the light fragments within the two-dimensional region are selectedfor evaluation. For example, light fragments 255A, 255B, 255C, and 255D,as projected in the light buffer 250, fall within the projected frustumregion 231 and are thus selected for further evaluation.

In an embodiment of step 125, the subset of light fragment within theprojected frustum region or an approximation thereof have their worldspace positions evaluated to determine if they are inside thethree-dimensional frustum. For example, step 125 further evaluates theworld space positions of light fragments 255A, 255B, 255C, and 255D todetermine if they fall within the three-dimensional frustum. If lightfragments lie within the frustum, then the shadow computation isperformed as described above. However, if a light fragment's world spaceposition is outside of the frustum, then the light fragment is ignoredfor purposes of the shadow computation.

In some cases, although a light fragment's projected position in thelight buffer may be within the projected frustum region; however, thelight fragment's world space position is outside of the frustum and thusdoes not contribute to the shadowing of a fragment of the eye buffer.FIG. 2E illustrates an example 275 of this case. In example 275, lightfragments 285 and 286 are projected into the light buffer at positions283 and 284, respectively. A fragment 280 from the eye buffer and alight source 281 define a frustum 279. Light fragments within thefrustum 279 lie between the fragment 280 and the light source 281 andthus may block some light from light source 281.

The projection of the frustum 279 into the light buffer plane 277defines region 282. Region 282 includes light fragment positions 283 and284. However, as can be seen from these fragments' world spacepositions, light fragment 285 is outside of the frustum 279, while lightfragment 286 is inside the frustum 279. Thus, light fragment 286contributes to the shading of fragment 280, while light fragment 285does not.

As discussed above, depth peeling can be used to provide improved shadowquality when rendering objects with complex light occlusion andscattering properties. Depth peeling creates several light buffers for alight source. The scene is rendered from the view of the light for eachlight buffer using a different near clipping plane. As a result, eachlight buffer includes a different set of fragments occluding or blockinglight from the light source.

FIG. 3 illustrates an example application 300 of depth peeling suitablefor use with an embodiment of the invention. In this example, the sceneis rendered from the position of the light source 305 to generate lightfragments for the light buffers 310 and 325.

For light buffer A 310, the scene is rendered with near clipping plane A315. Near clipping plane A 315 excludes, or clips, all objects betweenitself and the light source 305. As a result, light buffer A 310 willinclude light fragments from the outer portion of occluder 307. Forexample, light buffer A 310 can include entries 318A and 318B for lightfragments 320A and 320B, respectively.

For light buffer B 325, the scene is rendered with near clipping plane B330. Typically, near clipping plane B 330 is set to pass through theinterior of one or more occluders, such as occluder 307. Near clippingplane B 330 excludes, or clips, all objects between itself and the lightsource 305. As a result, light buffer B 325 will include light fragmentsfrom the interior portion of occluder 307. For example, light buffer B325 can include entries 333A and 333B for light fragments 335A and 335B,respectively.

Although light buffers 310 and 325 are shown in different positions forclarity, an embodiment of the invention uses the same image planelocation for both light buffers, changing only the location of theirrespective near clipping planes.

As discussed above, the light buffers 310 and 325 store attributes oflight fragments, including the light fragments' positions in a worldspace coordinate system and intensity values representing the amounts oflight received or blocked by the light fragments. The light fragments inlight buffer 310 represent light blocked by the exterior of occluder307, while the light fragments in light buffer 325 represent lightblocked by the interior of occluder 307. Each fragment in the eye bufferis evaluated against all of the light buffers associated with each lightsource in a manner similar to that described by method 100. As a result,the illumination values of the fragments of the eye buffer take intoaccount light blocked by the features in both the interior and exteriorof the occluder, which approximates the shadowing effects from occluderswith complex light occlusion and scattering properties. An embodimentcan use additional light buffers with near clipping planes passingthrough different interior portions of one or more occluders to provideeven better approximations of complex shadowing effects, at the cost ofincreased processing time and memory.

Many computer graphics systems include both a general purpose centralprocessing unit (CPU) as well as specialized graphics processing unit(GPU) adapted to render computer graphics. In some implementations, themethod of generating soft shadows can be executed primarily by the GPU.For example, the GPU can generate the eye buffer, an unshaded image, andone or more light buffers; evaluate light fragments for each fragment inthe eye buffer using a shader program; and blend illumination values ofthe fragments of the eye buffer with the unshaded image to create thefinal shadowed image.

In other implementations, the processing duties of the method ofgenerating soft shadows (and optionally other shading techniques) issplit between the GPU and CPU. In these implementations, the GPU is usedto generate unshaded or minimally shaded scenes. The final shading,including soft shadow generation, is deferred until after the scene iscompletely rendered and visibility of all fragments is determined. TheCPU can then be used to complete the shading operations, such as softshadow generation, on the visible fragments. The time required for thedeferred shading operations depends only on the resolution of the imageand the complexity of the shading operations. The geometric complexityof the underlying scene has no effect on the cost of the deferredshading operations. Deferred shading allows the GPU to concentrate onmaximizing scene geometry performance, while the CPU performs shadingoperations in roughly constant time based on the resolution of theimage.

FIG. 4A illustrates an implementation 400 of a method of generating softshadows according to an embodiment of the invention. In step 405, a GPUgenerates the eye buffer as described above. In a further embodiment,the GPU may simultaneously generate an unshadowed image from the cameraviewpoint for later blending with the illumination values. Many GPUssupport multiple render targets, which allow multiple image buffers withdifferent attribute values to be created simultaneously from the samescene geometry. Using this feature, the GPU can render the scene fromthe camera viewpoint to simultaneously create the eye buffer, whichstores fragment positions in world space, and the unshadowed imagebuffer, which can store color and optional attributes, such astransparency and/or depth values.

In step 410, the GPU generates one or more light buffers as describedabove. The GPU can generate multiple light buffers to support shadowsfrom multiple lights and/or multiple depth layers for each light.

In step 415, the CPU processes the eye buffer and one or more lightbuffers to create a shading buffer with illumination values for eachnon-background fragment of the eye buffer.

In step 420, the GPU blends the shading buffer with the unshadowed imagepreviously created to create a shadowed image.

Although the steps of method 400 are shown sequentially, embodiments ofmethod 400 can be executed in parallel and pipelined for multipleframes. FIG. 4B illustrates a table 450 illustrating an examplepipelined implementation of method 400. In execution phase 1, the GPUgenerates the eye buffer, light buffers, and unshaded image for a firstframe, frame 1. In execution phase 2, the CPU begins processing the eyebuffer and light buffers from frame 1 to create a shading buffer forframe 1. Additionally, the GPU generates the eye buffer, light buffers,and unshaded image for a second frame, frame 2, during execution phase2.

In execution phase 3, the GPU blends the unshaded image and shadingbuffer of frame 1 to create a shaded image for frame 1. Frame 1 is thenready for display or for further processing. Additionally, the GPUgenerates the eye buffer, light buffers, and unshaded image for a thirdframe, frame 3 during execution phase 3. During execution phase 3, theCPU processes the eye buffer and light buffers from frame 2 to create ashading buffer for frame 2.

In execution phase 4, the GPU blends the unshaded image and shadingbuffer of frame 2 to create a shaded image for frame 2. Frame 2 is thenready for display or for further processing. Additionally, the GPUgenerates the eye buffer, light buffers, and unshaded image for a fourthframe, frame 4, during execution phase 4. During execution phase 4, theCPU processes the eye buffer and light buffers from frame 3 to create ashading buffer for frame 3.

This example pipelined execution shown in FIG. 4B can be continued forany number of frames. Alternative pipeline execution arrangements canalso be used to accommodate different processing speeds of the GPU andCPU.

FIG. 5 illustrates details of an implementation 500 of a method ofgenerating soft shadows according to an embodiment of the invention.Implementation 500 includes a GPU 505, a CPU 510, and memory 515. In anembodiment, implementation 500 includes a unified memory architectureconfigured so that memory 515 is readily accessible to both the CPU 510and GPU 505 with little or no performance penalty. GPU 505 can beconfigured to create eye buffer 520, light buffers 525, and unshadowedimage buffer 530, as described above. CPU 510 can be configured toprocess eye buffer 520 and light buffers 525 as described above toproduce shading buffer 535. The shading buffer 535 can includeillumination values used to create soft shadows. In an additionalembodiment, the shading buffer 535 can also include values resultingfrom other lighting, shading, texturing, and other shading programoperations executed by the CPU 510. The GPU 505 can then blend shadingbuffer 535 with unshadowed image buffer 530 to create an image ready fordisplay or additional processing. Additionally, memory 515 can storebuffers for multiple frames simultaneously for use with pipelinedexecution arrangements as described above.

To facilitate efficient processing, several different additionaltechniques can be employed alone or together. First, if the CPU 510support SIMD instructions, one or more of the buffers can arrange datain a packed data format to facilitate the processing of multipleoperands simultaneously. In some implementations, the GPU 505 may becapable of outputting data for one or more of the buffers directly inthe packed data format. In other implementations, swizzling instructionsexecuted by the GPU 505 or CPU 510 can be used to rearrange data into anappropriate packed data format.

A second technique uses direct memory access (DMA) to transfer portionsof each buffer as needed between memory 515 and CPU 510. In anembodiment, two sets of DMA transfers are used to overlap computationswith data transferred. In this embodiment, data is transferred to and/orfrom one or more buffers in memory 515 to a first DMA buffer 540 in thecache or local storage of the CPU 510. In parallel with this DMAtransfer, the CPU 510 performs computations on the contents of a secondDMA buffer 545. When the computations and the DMA transfer are complete,the CPU begins processing data in the first DMA buffer 540 and a new DMAtransfer for DMA buffer 545 is initiated. The DMA transfer for buffer545 transfers processed data back to memory 515, for example into aportion of shading buffer 535, and retrieves additional data, such asdata from the eye buffer 520 and light buffers 525 for processing by theCPU 510.

The size of the DMA buffers 540 and 545 can be determined based upon theamount of data to be processed by the CPU 510, the DMA transfer speedand latency, and the amount of cache or local memory storage availablein the CPU 510. For example, each DMA transfer can include data for oneor more scanlines of a buffer.

FIG. 6 illustrates an example hardware system suitable for implementingan embodiment of the invention. FIG. 6 is a block diagram of a computersystem 1000, such as a personal computer, video game console, personaldigital assistant, or other digital device, suitable for practicing anembodiment of the invention. Computer system 1000 includes a centralprocessing unit (CPU) 1005 for running software applications andoptionally an operating system. CPU 1005 may be comprised of one or moreprocessing cores. Memory 1010 stores applications and data for use bythe CPU 1005. Storage 1015 provides non-volatile storage forapplications and data and may include fixed disk drives, removable diskdrives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, orother optical storage devices. User input devices 1020 communicate userinputs from one or more users to the computer system 1000, examples ofwhich may include keyboards, mice, joysticks, touch pads, touch screens,still or video cameras, and/or microphones. Network interface 1025allows computer system 1000 to communicate with other computer systemsvia an electronic communications network, and may include wired orwireless communication over local area networks and wide area networkssuch as the Internet. An audio processor 1055 is adapted to generateanalog or digital audio output from instructions and/or data provided bythe CPU 1005, memory 1010, and/or storage 1015. The components ofcomputer system 1000, including CPU 1005, memory 1010, data storage1015, user input devices 1020, network interface 1025, and audioprocessor 1055 are connected via one or more data buses 1060.

A graphics subsystem 1030 is further connected with data bus 1060 andthe components of the computer system 1000. The graphics subsystem 1030includes a graphics processing unit (GPU) 1035 and graphics memory 1040.Graphics memory 1040 includes a display memory (e.g., a frame buffer)used for storing pixel data for each pixel of an output image. Graphicsmemory 1040 can be integrated in the same device as GPU 1035, connectedas a separate device with GPU 1035, and/or implemented within memory1010. Pixel data can be provided to graphics memory 1040 directly fromthe CPU 1005. Alternatively, CPU 1005 provides the GPU 1035 with dataand/or instructions defining the desired output images, from which theGPU 1035 generates the pixel data of one or more output images. The dataand/or instructions defining the desired output images can be stored inmemory 1010 and/or graphics memory 1040. In an embodiment, the GPU 1035includes 3D rendering capabilities for generating pixel data for outputimages from instructions and data defining the geometry, lighting,shading, texturing, motion, and/or camera parameters for a scene. TheGPU 1035 can further include one or more programmable execution unitscapable of executing shader programs.

The graphics subsystem 1030 periodically outputs pixel data for an imagefrom graphics memory 1040 to be displayed on display device 1050.Display device 1050 is any device capable of displaying visualinformation in response to a signal from the computer system 1000,including CRT, LCD, plasma, and OLED displays. Computer system 1000 canprovide the display device 1050 with an analog or digital signal.

In embodiments of the invention, CPU 1005 is one or more general-purposemicroprocessors having one or more processing cores. Further embodimentsof the invention can be implemented using one or more CPUs withmicroprocessor architectures specifically adapted for highly paralleland computationally intensive applications, such as applicationsproviding real-time interactive computer graphics. FIG. 7 illustrates anexample processor 2000 suitable for implementing an embodiment of theinvention.

Processor 2000 includes a number of processor elements, each capable ofexecuting independent programs in parallel. Processor 2000 includes PPEprocessor element 2005. PPE processor element is a general-purposeprocessor of CISC, RISC, or other type of microprocessor architectureknown in the art. In one example, PPE processor element 2005 is a64-bit, multithreaded RISC architecture microprocessor, such as thePowerPC architecture. PPE processor element 2005 can include a cachememory 2007 partitioned into one, two, or more levels of cachestemporarily holding data and instructions to be executed by PPEprocessor element 2005.

For additional performance, processor 2000 includes a number of SPEprocessor elements 2010. In this example, processor 2000 includes eightSPE processor elements 2010A-2010H; however, other example processorscan include different number of SPE processor elements. SPE processorelements 2010 are adapted for stream processing of data. In streamprocessing, a program is executed repeatedly on each item in a large setof data. To facilitate stream processing, the SPE processor elements2010 may include instruction execution units capable of executing SIMDinstructions on multiple data operands simultaneously. SPE processorelements 2010 may also include instruction units capable of executingsingle-instruction, single data (SISD) for more general processingtasks.

Each SPE processor element, such as SPE processor element 2010A,includes local data and instruction storage 2012A. Data and instructionscan be transferred to and from the local data and instruction storage2012A via DMA unit 2014A. The DMA units, such as unit 2014A, are capableof transferring data to and from each of the SPE processor elements 2010without processor supervision, enabling the SPE processor elements 2010to process data continuously without stalling.

Data and instructions are input and output by the processor 2000 viamemory and I/O interfaces 2015. Data and instructions can becommunicated between the memory and I/O interfaces 2015, the PPEprocessor element 2005, and SPE processor elements 2010 via processorbus 2020.

Further embodiments can be envisioned to one of ordinary skill in theart from the specification and figures. In other embodiments,combinations or sub-combinations of the above disclosed invention can beadvantageously made. The block diagrams of the architecture and flowcharts are grouped for ease of understanding. However it should beunderstood that combinations of blocks, additions of new blocks,re-arrangement of blocks, and the like are contemplated in alternativeembodiments of the present invention.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

1. A computer-implemented method of creating a computer graphics image,the method comprising: creating, in a computer memory, an objectfragment representing a portion of an object in a scene from a virtualcamera viewpoint, wherein the object fragment includes a visible surfaceposition in the scene; creating, in the computer memory, a set of lightfragments for a light source, the set of light fragments representing aportion of the scene from a viewpoint associated with the light source,wherein each light fragment includes a position in the scene; defining,using a computer processor operatively connected to the computer memory,a frustum between the visible surface position of the object fragment inthe scene and the light source, the frustum including a shape with anapex located on the visible surface position of the object fragment anda base defined by a size and a position of the light source; selecting,using the computer processor, a portion of the set of light fragmentshaving positions within the frustum; determining, using the computerprocessor, a shading value for the object fragment from the selectedportion of the light fragments; determining, using the computerprocessor, an unshaded visual attribute value for the object fragment;and altering, using the computer processor, the unshaded visualattribute value of the object fragment according to the shading value ofthe object fragment, wherein altering the unshaded visual attributevalue of the object fragment comprises scaling the unshaded visualattribute value of the object fragment according to the shading value ofthe object fragment.
 2. The method of claim 1, wherein the shading valuerepresents a light intensity received by the object fragment from thelight source.
 3. The method of claim 1, wherein the shading valuerepresents a light intensity and a light color received by the objectfragment from the light source.
 4. The method of claim 1, wherein:creating the object fragment comprises rendering at least the objectfrom the camera viewpoint; and creating the set of light fragmentscomprises rendering the portion of the scene from the viewpointassociated with the light source.
 5. The method of claim 4, whereinrendering the portion of the scene from the viewpoint associated withthe light source comprises: rendering the portion of the scene using afirst near clipping plane to create a first subset of light fragmentsand rendering the portion of the scene using a second near clippingplane to create a second subset of light fragments.
 6. The method ofclaim 1, wherein selecting the portion of the set of light fragmentshaving positions within the frustum comprises: projecting the frustuminto an image plane associated with the set of light fragments todetermine a two-dimensional frustum region; defining an approximateprojected frustum region that approximates but is not exactly thedetermined two-dimensional frustum region; selecting a second portion ofthe set of light fragments having projected positions in the image planewithin the approximate projected frustum region; evaluating the positionin the scene of each of the second portion of the set of light fragmentswith respect to the frustum to determine whether each of the secondportion of the set of light fragments is within the frustum.
 7. Themethod of claim 1, wherein the shading value includes a sum of attributevalues associated with the selected portion of the set of lightfragments.
 8. The method of claim 1, wherein the shading value includesa product of attribute values associated with the selected portion ofthe set of light fragments.
 9. The method of claim 1, wherein each ofthe set of light fragments includes an attribute value indicating theamount of light blocked from the light source by an occluder representedby the light fragment.
 10. The method of claim 9, wherein the attributevalue is determined at least in part by a color attribute of theoccluder represented by the light fragment.
 11. The method of claim 9,wherein the attribute value is determined at least in part by a distancebetween the light source and the light fragment.
 12. The method of claim9, wherein the attribute value is determined at least in part by atransparency attribute of the occluder represented by the lightfragment.
 13. A method of creating a computer graphics image, the methodcomprising: creating, with a graphics processing subsystem including atleast one graphics processing unit, a first buffer including objectfragments representing a scene from a first viewpoint of a first frame,wherein the first buffer includes visible surface positions of objectfragments in the scene; creating, with the graphics processingsubsystem, a second buffer including light fragments representing ascene from a second viewpoint associated with a light source, whereinthe second buffer includes positions of light fragments of the scene;defining, for each of at least a portion of the object fragments of thefirst buffer, a frustum between each of the visible surface positions ofobject fragments of the portion and the light source, each frustumincluding a shape with an apex located at the visible surface positionof one of the object fragments of the portion and a base defined by asize and a position of the light source; selecting, for each of at leastthe portion of the object fragments of the first buffer, light fragmentsin the second buffer having positions within each corresponding frustum;determining, for each of the object fragments of the portion of thefirst buffer, a shading value from the corresponding selected lightfragments; storing the shading values in a third buffer; creating, withthe graphics processing subsystem, a fourth buffer including objectfragments representing a scene from the first viewpoint of the firstframe, wherein the fourth buffer includes unshaded visual attributevalues of object fragments in the scene; and altering the unshadedvisual attribute values of the fourth buffer according to the shadingvalues of the third buffer.
 14. The method of claim 13, whereindefining, selecting, determining, and storing are performed by thegraphics processing subsystem.
 15. The method of claim 13, whereindefining, selecting, determining, and storing are performed by a centralprocessing unit in communication with the graphics processing subsystem.16. The method of claim 15, further comprising: creating, with thegraphics processing subsystem, a fifth buffer including object fragmentsrepresenting a scene from a first viewpoint of a second frame, whereinthe fifth buffer includes positions of object fragments in the scene;and creating, with the graphics processing subsystem, a sixth bufferincluding light fragments representing a scene from a second viewpointof the second frame associated with a light source, wherein the sixthbuffer includes positions of light fragments of the scene; whereindefining, selecting, determining, and storing are performed in parallelwith creating the fifth and sixth buffers.
 17. The method of claim 13,wherein altering the unshaded visual attribute values of the fourthbuffer according to the shading values of the third buffer comprisesblending the third and fourth buffers.
 18. The method of claim 13,wherein the shading values of the third buffer represent lightintensities received by the scene fragments from the light source. 19.The method of claim 13, wherein the shading values of the third bufferrepresent light intensities and light colors received by the scenefragments from the light source.
 20. The method of claim 13, whereinselecting, for each of at least the portion of the scene fragments ofthe first buffer, light fragments in the second buffer comprises: foreach corresponding frustum, projecting the frustum into an image planeassociated with the second buffer to determine a two-dimensional frustumregion; for each corresponding frustum, defining a bounding rectangleapproximating but not equaling the shape of the projected frustum; foreach bounding rectangle, selecting a portion of the light fragments inthe second buffer having projected positions in the image plane withinthe bounding rectangle; and for each of the selected portion of thelight fragments associated with each bounding rectangle, evaluating theposition in the scene of each light fragment with respect to thecorresponding frustum to determine whether the light fragment is withinthe corresponding frustum.
 21. The method of claim 13, wherein each ofthe shading values of the third buffer include a sum of attribute valuesassociated with the selected portion of the set of light fragments. 22.The method of claim 13, wherein each of the shading values of the thirdbuffer include a product of attribute values associated with theselected portion of the set of light fragments.
 23. The method of claim13, each of the light fragments includes an attribute value indicatingthe amount of light blocked from the light source by the light fragment.24. The method of claim 23, wherein the attribute value is determined atleast in part by a color attribute of the light fragment.
 25. The methodof claim 23, wherein the attribute value is determined at least in partby a distance between the light source and the light fragment.
 26. Themethod of claim 23, wherein the attribute value is determined at leastin part by a transparency attribute of the light fragment.
 27. Acomputer-implemented method of creating a computer graphics image, themethod comprising: determining, using a computer processor, sceneattributes for an object fragment, the scene attributes includingcoordinates of a first image space visible surface position of theobject fragment and an opacity value of the object fragment; storing, ina computer memory operatively connected to the computer processor, thescene attributes in an eye buffer in a packed data format; determining,using the computer processor, light attributes for a set of lightfragments of a light source, the light attributes including coordinatesof a second image space position of the location of the respective lightfragments; storing, in the computer memory, the light attributes in alight buffer in a packed data format; defining, using the computerprocessor, a frustum between the first image space visible surfaceposition of the object fragment and a second image space position of thelight source, the frustum including a shape with an apex located on thevisible surface position of the object fragment and a base defined by asize and a position of the light source; selecting, using the computerprocessor, a portion of the set of light fragments having coordinateswithin the frustum; determining, using the computer processor, anintensity value for each selected portion of the set of light fragments;determining, using the computer processor, an illumination value for theobject fragment, the illumination value being determined by therespective intensity values; determining, using the computer processor,a shaded value for the object fragment based on the illumination valueand a color value; and rendering, on a display operatively connected tothe computer processor, the computer graphics image using the determinedshaded value for the object fragment.